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ABSTRACT 

This project provided feedback on the effect of the 
investment made in Information and Communication Technology (ICT) access and 
provision in schools in Scotland. The project examined the levels of 
performance of pupils at Primary 7 and Secondary 4 on a range of ICT skills 
and abilities, their experiences of and attitudes to ICT and, the views of 
their teachers. Two national surveys of pupils and their teachers were 
undertaken in 1998-99 and 2000-01. Three forms of data collection were used: 
questionnaires to pupils, teachers, and school. ICT coordinators; a series of 
written booklets designed to assess pupils* knowledge and understanding of 
ICT and its uses; and visits to a sub-sample of schools to assess pupils* 
skills and understanding as they worked through a series of practical tasks 
using a computer. This report presents results in the following areas: (1) 

the extent to which pupils are able to determine how they will tackle a task; 
.(2) pupils* access to e-mail; (3) impact of ICT guidelines on primary- 
secondary curriculum continuity; (4) how developing electronic communications 
with others outside the school could be used to enhance the range of teaching 
and leaning experiences; (5) the role of ICT in the classroom of the future; 
and (6) how teachers can capitalize on students* access to ICT at home. (MES) 
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introduction 

The Impact of Information and Communication Technology Initiatives in Scottish 
Schools project was funded by the Scottish Executive Education Department to provide 
feedback on the effect of the considerable investment made in Information and 
Communication Technology (ICT) access and provision in schools. More specifically, the 
project attempted to determine the levels of performance of pupils at Primary 7 (P7) and 
Secondary 4 (S4) on a range of ICT skills and abilities, their experiences of, and attitudes 
to, ICT and, in addition, the views of their teachers. As a result, two national surveys of 
pupils and their teachers were undertaken, the first in 1998-99 and the second in 
2000 - 01 . 

In 2000, Interchange 63 was published by the Scottish Executive, setting out the origins 
of the project and its aims as well as the main findings from the first phase (1998-99). 
This Insight reports on the findings from the second phase in 2000-01, comments on the 
impact to date and makes recommendations for future development in the area of ICT. 

The Study 

In both phases the study focused on pupils in P7 and S4 and their teachers. In 2000-01, 
1332 P7 pupils in 72 primary schools and 1409 S4 pupils in 80 secondary schools 
participated in the project. The two year interval between phases meant that it was 
possible to track some of the P7 pupils from 1998-99 and assess them again in 2000-01 
when they were in Secondary 2 (S2). Within each phase three forms of data collection 
were used. The first consisted of questionnaires to pupils, teachers and school ICT co- 
ordinators. These were distributed at the same time as the series of written booklets 
designed to assess pupils' knowledge and understanding of ICT and its uses. The third 
form of data gathering involved visiting a sub-sample of schools and assessing pupils' 
skills and understanding as they worked through a series of practical tasks, using a 
computer. In 2000-01, the written element of the study (questionnaires and written 
booklets) took place during February to April 2001, followed by the practical component 
in May and June 2001. 

Pupils’ ICT Knowledge and Skills 

The assessment framework (written booklets and practical tasks) consisted of three 
categories of performance: Knowledge and Understanding, Personal Appropriation and 
Practical Competence. Within each category, four sub-categories were identified: 
Hardware, Software, Communications and Networks and Uses and Impact of ICT. The 
majority of tasks were concerned with computers, although a few included questions on 
other forms of ICT. 

Knowledge and Understanding 

In Knowledge and Understanding of Hardware, tasks typically asked pupils to name parts 
of computers and their peripherals, to identify desktop icons and to know about viruses, 
icons, networks and a range of disks. So ft wo re tasks focused on using software for 
specific tasks, including graphics, databases, spreadsheets, CD-ROMs, and recognising the 
more common icons on software toolbars. The example No Smoking required pupils to 
•'^''nnise various aspects of a word processing package. 



Example!: no Smoking 
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Andy was writing up a project on the dangers of smoking. The picture shows the 
screen of his computer. 
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Why you should not smoke.qi 




Smoking is a danjerus thing todo becos it 
can kill you. qi 






Andy's spelling is not very good. What could he use from the software to help him 
get his spelling correct? 



Look at the picture and say what the font type and the font size are. 

The font type is 

The font size is 

Look at the picture below of part of the toolbar. Choose a letter from the box to 
show what each icon will do. Write the letter in the box beside the icon. (You will not 
need all the letters): 



Open file - 0 


5ave - 5 


Print - P 


Cut - X 


Copy - C 


Insert text - 1 



The first one has been done for you. 
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A total of seven marks was available for this task. On average, P7 pupils scored 5.2 marks 
and 54 pupils scored 5.4. 



At P7, attainment was greatest in Knowledge and Understanding of Hardware an6 
Software, with scores of 63% and 70% respectively for the sample. The 54 papers 
contained more tasks, and more demanding ones, although the majority of tasks were 
common to both stages. At 54, pupil performance levels were also better on Hardware 
(71%) and Software (72o/o). 

Within Knowledge and Understanding, tasks on Communications and Networks expected 
knowledge of e-mail, video-conferencing, internet search engines and associated 



terminology such as URL and links. Uses and Impact of ICT tasks focused on the ways in 
which technology is used in society, including the kinds of tasks that workers in ICT 
might undertake. On Communications and Networks, P7 pupils scored 53% of the 
available marks and 70% on Uses and Impact of ICT ; at S4, the figures were 61% and 
58% respectively. 



Personal Appropriation 

Personal appropriation, the ability to identify and use appropriate strategies to complete 
a task, was also assessed through the written papers. At P7, the success rates were 49% 
for Hardware, 33% for Software and 35% for Communications and Networks. At S4, the 
corresponding figures were 58%, 61% and 45%. 

Example 2: For Sale 



Jodie is going to buy a second hand computer. She saw this advert for a computer in 
the local newspaper: 




1. Jodie thinks that the computer can be used to access the Internet. 
Is she correct? 

Give a reason for your answer. 



2. The multimedia software which she is going to buy needs 48Mb of 
memory. 

Will she be able to run this software on the computer? 

How do you know? 



In this task. For Sale, a total of four marks was available. At P7, the average score was 
1.6 and at S4 it was 2.4. 



To what extent are 
your pupils able to 
determine how 
they will tackle a 
particular task? 

Are they usually 
directed to specific 
applications or 
resources? 
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What opportunities 
do your pupils 
have to e-mail 
others in school? 
How many of them 
have access to 
e-mail and the 
internet outwith 
school? 



What aspects 
would you expect 
to develop most 
across the 10-14 
age range? What 
impact will the 
introduction of the 
5-1 4 ICT 
Guidelines have 
upon primary- 
secondary 
curriculum 
continuity? 



Practical Competence 

Incompatibility of equipment, platforms and applications resulted in a smaller data 
gathering exercise for this aspect than was originally anticipated: 198 P7 pupils and 195 
S4 pupils tackled the practical assessment tasks. Assessments were made on a one-to- 
one basis and the instructions were presented in written form and read out by the 
assessor. 

At both stages, pupil performance was most secure on word processing. Basic procedures 
for spreadsheets were fairly secure. Competence levels on e-mail and using the internet 
were difficult to establish accurately due to a lack of available facilities in many schools, 
but where pupils did have access, they accomplished most of the tasks satisfactorily. 

Comparing Primary 7 and Secondary 4 

Thirty-six tasks were common to both stages. The pattern of S4 superiority observed in 
the first phase was repeated in the second phase, with the older pupils achieving higher 
success rates on 28 tasks while P7 were better on eight tasks. There was no clear pattern 
to the differences in terms of the kinds of tasks where one group was more successful. 

Comparing 1999 and 2001 findings 

There was no evidence of improved performance at P7 although there did appear to be 
some evidence of improvement at S4, primarily due to greater understanding of software 
applications and procedures. 

The Perceptions of School Staff and Pupils 

Questionnaires were issued to the pupils involved in the survey, their teachers and the ICT 
co-ordinators within their schools. The co-ordinators have the main responsibility for 
planning the implementation of the various initiatives within schools. 

The ICT Co-ordinators 

Remits 

Co-ordinators in both sectors indicated that their remit was escalating in scope and 
volume. New elements had been added in response to local authority and government 
initiatives, and the publication of guidelines for ICT had had consequences for their 
workloads. Dealing with technical problems occupied a great deal of their time. 

Sources of advice and information 

Both primary and secondary co-ordinators reported that their main sources of support 
were local authority staff (68% primary; 59% secondary), and colleagues in their own 
school (41% primary; 59% secondary). The most valued sources of information were 
mainly within the mainstream educational system - their own schools, the local authority 
and, for primary teachers. Learning and Teaching Scotland. Although there was a 
reported increase in the use of the internet, of teachers' networks and the Scottish 
Virtual Teachers Centre, as in the previous phase these were consulted by relatively few 
(< 20 %). 

The development of policy and planning 

Both sectors indicated that a lack of teacher confidence and/or expertise was hindering 
the implementation of the 5-14 Guidelines for ICT. Other obstacles included the time to 
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teach, finance for hardware and other resources, and the monitoring of progress. The 
majority of schools in both sectors had written policies for ICT; approximately half of 
the secondary schools based this on Using the Superhighways: ICT and Development 
Planning. 

In general, communications about ICT within schools appeared to have become more 
effective since 1999, primarily through in-school staff development activities and 
informal contact. More use was being made of e-mail in secondary schools for this 
purpose. 

Progress with priorities 

Primary co-ordinators indicated that their priority had been resourcing for ICT, e.g. 
installing hardware, and the data showed no clear pattern of development in learning 
and teaching in primary schools. The most securely established practice remained the 
use of CD-ROM resources in primary classrooms (70%), and this was one area where 
pupils did appear to have gained in expertise since the first phase. There was little or 
no change in administrative usage, e.g. enhancing the quality of pupil reports (48%) 
and centralising the assessment and test results of pupils (25%). 

In secondary schools, advances in a range of learning and teaching strategies were 
reported. These included supporting pupils with learning difficulties (from 56% in 
1999 to 74% in 2001) and securing opportunities for internet access (29 to 74%). 
Administratively, the quality of reports had been enhanced (48 to 60%) and 
assessment and testing records had been centralised by a greater proportion of schools 
(8 to 30o/o). 

Using ICT to communicate with others outside the school - parents, professional 
colleagues, pupil peer groups or the local community - remained the least well 
advanced in both sectors. 

Increased benefits 

In both sectors, co-ordinators identified a number of benefits from ICT including 
aspects of management activity, increased pupil motivation, improved presentation 
and increased ICT skills amongst pupils and teachers. 

The Primary Teachers 

Of the primary teachers who responded, 81% had a computer at home and 77% of 
these were linked to the internet. The computer was used most frequently by our 
overall sample of teachers for word processing for school (78%), internet searches 
(65%), using e-mailing for personal (64%) and professional (37%) purposes, and CD- 
ROMs (62o/o). 

Working with pupils 

In school, 93% of primary teachers had at least one computer in their classroom 
regularly in use by pupils and 54% also used a school ICT base, suite or other shared 
computer facilities. 

The only significant change in pupil use of school computers since 1999 was an 
increased use of the internet to search for information (from 3% in 1999 to 27% in 
2001). Teachers reported that their pupils showed a higher level of independence from 
the teacher when working on the computer (63%) while 45% said that they had 



How could 
developing 
electronic 
communications 
with others outside 
the school be used 
to enhance the 
range of teaching 
and learning 
experiences? 



Do you have a 
vision of the 
classroom of the 
future? What role 
does ICT play in 
this? 
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evidence that pupil attainment had improved and 94% that ICT had contributed in 
significant ways to the learning of their pupils. 

More teachers were using ICT to create curriculum materials (43%) or download materials 
from the internet (26%). There was little change in the proportion using ICT to assist 
high attaining pupils or for providing differentiated activities, although the proportion 
giving specialist assistance to pupils with learning disabilities increased from 34 to 47%. 

Administrative and management tasks 

In terms of management activity, 41% were using ICT to produce pupil records for 
parents, with around 17% using it for managing other types of assessment data. 

However, significant percentages indicated that producing records for parents (22%), 
making profiles of pupil progress (29%), recording and analysing test results (46%) and 
setting up systems that would allow pupils to use ICT to record and monitor their own 
attainments (54%) were unlikely to be developed in the foreseeable future. 

Fifty-five percent of primary teachers thought that the classroom of the future would be 
very different from the present while 40% considered that the importance of ICT had 
been greatly exaggerated. 

The Secondary Teachers 

Of the secondary teachers, 88% had a computer at home and 75% of these were linked 
to the internet. The computer was used most frequently by the secondary teachers for 
the same purposes as their primary colleagues. 

Working with pupils 

In school, 62% had at least one computer in their classroom regularly in use by pupils 
while 48% also used a school ICT base, suite, lab or other shared computer facilities. 

More teachers were using ICT to create curriculum materials (46%) or to download 
materials from the internet (25%). Increased percentages were using ICT to assist high 
attaining pupils (40%) and for providing differentiated activities (44%); the proportion 
giving specialist assistance to pupils with learning disabilities had increased from 15% 
to 24%. 

Little change was reported in pupil use of ICT except for using the internet to search for 
information (from 8% in 1999 to 25% in 2001) and using CD-ROMs (19 to 28%). 

However, 66% thought that pupils showed a higher level of independence from the 
teacher when working on the computer while 41% said that they had evidence of 
improved attainment and 84% agreed that ICT had contributed in significant ways to 
pupils' learning. 

Administration and management tasks 

In terms of administration and management, 28% were using ICT for pupil records for 
parents and 20% for other types of assessment data management. Significant 
percentages indicated that producing records for parents (27%), making profiles of pupil 
progress (29%), recording and analysing test results (30%) and setting up systems that 
would allow pupils to use ICT to record and monitor their own attainments (73%), were 
unlikely in the foreseeable future. 

Just over half (55%) envisaged that the classroom of the future would be very different 
from the present while 49% thought that the importance of ICT had been greatly 
exaggerated. 




The Pupils 

The responses from 2001 were remarkably similar to those of 1999 and showed little 
variation with age. Where there was an age difference, the S2 figures fell between those 
for P7 and S4. Just over half of the pupils in each group indicated that they knew 
'enough (about computers) to get by', while around one third indicated they knew 'a lot 
or were real experts'. 51% (P7) and 65% (S4) regarded learning to use computers as 'very 
important'; almost all the remainder regarded it as 'quite important'. 

In school 

At school, 55% of S4 pupils used a computer three or more times a week, while most of 
the younger pupils used it once or twice a week or less. The range of typical uses in 
school had changed little, e.g. word processing, drawing and using CD-ROMs. The only 
changes were increases in the use of the internet and e-mail. 

More than half the S4 pupils studying subjects where computers were used regularly (e.g. 
Business Management, Graphic Communication or Accounting) had used them in the 
previous week, while 25o/p of pupils had used them in English and/or Art and Design. The 
figures for computer use in all other subjects were very low, showing little change since 
1999. 

The majority of pupils in each age group indicated that they liked using computers 
because the work was more interesting (86% P7 and 78% S4) and the product neater 
(75% P7 and 90% S4). They liked being able to find information not accessible through 
books (76% P7 and 68% S4) and that they got to use their own ideas and imagination 
(58% P7 and 52% S4). Fewer than 10% of pupils in all three age groups thought that 
the computer-based work in school was too hard while 39% at P7, 33% at S2 and 17% 
at S4 thought it too easy. 

At home 



Across the stages, 80% at P7 and 83% at S4 indicated that they had access to a 
computer outside school and half used it on three or more days a week. Of the total 
pupil sample, 73% at P7 and 77% at S4 had a computer in their own home and for 57% 
of P7 and 60% of S4 pupils this was linked to the internet. Some had greater access, 
with 26% at P7 and 33% at S4 having a computer in their own room. 

There were a number of differences in the usage at home and school, with the most 
significant (educationally) being the lack of pressure of time at home (78% P7 and 87% 
S4 of those with access indicated that they could spend as long as they liked doing their 
work on the computer at home, while at school the corresponding figures were 6% and 
12%). In addition, they were able to be more adventurous at home with 89% of P7 and 
92% of S4 pupils indicating that they experimented with different applications and 
activities, while at school the corresponding figures were 42% and 32%. 



How can teachers 
capitalise on the 
access at home to 
ICT enjoyed by so 
many pupils? How 
can they support 
those who do not 
have such access? 



Summary 

This study was set up to provide baseline data at the introduction of a series of ICT. 
initiatives in Scottish schools and to consider, two years later, the extent of the impact of 
these initiatives. This has now been completed and the findings are complex. The 
questionnaires have yielded evidence that the use of ICT both in and out of school is 
increasing, with a substantial majority of pupils and teachers having access to a range of 
technologies, particularly computers, most of which are connected to the internet. The 
uses to which they were put varied somewhat, with teachers tending to use word 



processing and other 'task-oriented' programmes while pupils were more creative and 
'experimented' with the technology. 

Attitudes to ICT were, in the main, positive although a significant proportion of teachers 
thought that the benefits to the classroom had been greatly exaggerated. Technical 
problems and inconsistencies across platforms and applications were sources of 
frustration for both teachers and pupils. An additional frustration for many teachers was 
the extent to which staff development in ICT had not kept pace with technological 
developments. 

Pupils' performance levels were little changed across the two surveys, with considerable 
consistency across stages and categories of assessment. Increases were noted in those 
aspects which supported 'research,' i.e. using the internet and CD-ROMs. The extent to 
which teachers considered that some pupils were more advanced than most of their 
teachers, and indicated concern about such a 'skills gap,' decreased over the period of the 
study but remained a significant issue. It may be that the time pupils spend 'playing' 
with the different uses and applications is, in part, responsible for this perceived gap. A 
further issue of concern is the extent to which those pupils who do not have access to 
computers at home may be disadvantaged in comparison with those who do. 

One of the tensions in the implementation of ICT initiatives lies in the debate as to 
whether ICT should be considered a subject in its own right or as a means to learning 
within other subjects and contexts. The development of ICT suites or labs has tended to 
support the former approach, with timetabled sessions where the focus is on ICT skill 
acquisition. The measures of achievement used within this study focused on ICT 
knowledge and skills per se. 

An alternative approach considers that the potential of ICT lies in the ways in which it 
can make a range of information accessible to pupils studying, for example, science, art, 
the social subjects or modern languages, and facilitate their learning. The Assessment of 
Achievement Programme, funded by SEED, is currently piloting an alternative approach 
to the assessment of ICT knowledge and skills. Tasks have been developed to provide 
evidence of ICT competence within the context of a national survey of performance in 
Social Subjects 5-14. Assessing skills and understanding within relevant and interesting 
contexts should help to foster the view of ICT as a source of support in learning and 
teaching more generally. The findings from the survey should contribute considerably to 
this debate. 



The full report. The Impact of ICT Initiatives in Scottish Schools: Final Report, is available 
from Dr Rae Condie, Faculty of Education, University of Strathclyde (price £7.00). 



1. Classroom Assistants: Key Issues from the National Evaluation 

2. The Impact of ICT Initiatives in Scottish Schools 



If you have views on Insight or wish to find out more about SEED'S research and economics 
programme, please contact the Research, Economic and Corporate Strategy Unit, Scottish 
Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ. 
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